Bringing to Life the Musical Properties of the Eyes

Art, music, and science come together as computer musicians and computer scientists collaborate to create EyeMusic, a system for composing and performing eyecontrolled musical and multimedia compositions. In EyeMusic, an eye-tracking device communicates with a musical and multimedia authoring environment to enable composers and artists to sonify and visualize eye movement data. During a live performance, the performer moves his or her eyes to interact with visual stimuli and, in doing so, triggers a range of musical and visual elements. Two major challenges in designing eye-controlled multimedia software include (a) composing pieces that use the noise and error in eye movements and eye trackers for an expressive and artistic outcome and (b) finding a balance between having the performer directly control that outcome and having the composition proceed on its own while responding in meaningful and interesting ways to eye movements. EyeMusic compositions have been selected by multiple juries for live performance at major computer music conferences.